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CURRENT  STATE  OF  ARMY  AVIATOR  SELECTION 
INTRODUCTION 

The  purpose  of  this  research  is  to  document  the  main  findings 
and  conclusions  of  analyses  performed  to  assess  the  current  state 
of  Army  aviator  selection. 

Problem 


The  predictive  validity  of  the  Army  Flight  Aptitude  Selection 
Test  (FAST)  declined  from  .31  in  1988  to  .17  in  1993  and  a  further 
decline  is  inevitable.^  This  problem  is  exacerbated  by  the  fact 
that  no  alternate  forms  of  the  FAST  are  available  that  can  be 
employed  in  the  event  that  the  FAST  is  seriously  compromised. 
During  times  when  research  and  development  resources  were  more 
plentiful,  evidence  of  the  FAST's  declining  predictive  validity 
would  have  been  sufficient  justification  for  the  immediate 
initiation  of  a  research  and  development  program  to  produce  a  new, 
improved  aviator  selection  test  battery.  In  the  present  climate 
of  severely  limited  research  and  development  resources,  however, 
there  is  a  need  to  examine  carefully  the  consequences  of  the 
FAST'S  declining  predictive  validity  before  it  is  possible  to 
identify  the  most  practical  remedial  measures  and  to  argue 
convincingly  for  their  implementation. 

The  most  apparent  consequences  of  an  ineffective  selection 
test  are  (a)  a  high  rate  of  eliminations,  a  high  rate  of  setbacks^ 
or  both,  and  (b)  the  dollar  losses  and  inefficiencies  associated 
with  eliminations  and  setbacks.  Another  possible  consequence  of 
an  ineffective  selection  test  is  the  graduation  of  a  larger  number 
of  students  whose  flying  skills  are  marginal .  Although  it  was 
possible  to  examine  both  eliminations  and  setbacks,  it  was  beyond 
the  scope  of  this  study  to  assess  trends  in  the  average  skill 
level  of  Army  aviators  at  the  time  they  joined  an  operational  unit 
or  their  effectiveness  thereafter. 

Obi ectives 


One  of  the  primary  objectives  of  this  project  was  to  compile 
data  on  the  number  of  students  who  have  been  eliminated  or 'set 
back  because  of  the  types  of  skill  deficiencies,  knowledge 
deficiencies,  or  behavioral  problems  that,  in  principle,  are 


^  The  original  FAST  has  been  modified  twice  since  it  was  originally  validated 
in  1963.  The  first  and  second  revisions  are  referred  to  as  the  Revised  Flight 
Aptitude  Selection  Test  (RFAST)  and  the  Alternate  Flight  Aptitude  Selection 
Test  (AFAST) ,  respectively.  The  1988  validation  was  based  on  RFAST  scores  and 
the  1993  validation  was  based  on  AFAST  scores. 

^  The  term  "setback"  refers  to  a  training  event  in  which  a  student  is  removed 
from  his  or  her  initial  training  class  and  placed  in  a’ class  that  commenced 
training  at  a  later  time.  Ordinarily,  the  purpose  of  a  setback  is  to  enable  a 
student  to  repeat  a  portion  of  the  flight  training  program.  However,  in  the 
event  of  absence  from  training  because  of  illness  or  a  personal  problem,  a 
setback  may  enable  the  student  to  reenter  training  at  the  point  in  the  program 
at  which  the  student's  absence  commenced. 
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measurable  by  a  selection  test  battery.  When  compiling  data  on 
eliminations  and  setbacks,  it  was  observed  that  some  of  the  causes 
given  for  eliminations  and  setbacks  were  ambiguous  or  misleading. 
This  observation  led  to  the  establishment  of  a  related  objective: 
Conduct  a  detailed  study  of  the  types  and  frequencies  of  the 
causes  cited  (by  instructors)  for  eliminations  and  setbacks. 

Another  major  objective  was  to  determine  the  probable 
benefits  of  increasing  the  FAST  cut-score.  Specifically,  the 
objective  was  to  examine  the  distribution  of  FAST  scores  for  three 
groups  of  students  and  to  determine  the  number  in  each  group  that 
would  have  been  excluded  if  the  cut- score  had  been  greater  than 
the  current  value  of  90.  The  three  groups  of  students  include  (a) 
students  who  were  eliminated,  (b)  students  who  completed  training 
but  experienced  one  or  more  setbacks,  and  (c)  students  who  were 
not  eliminated  from  training  and  who  completed  training  without  a 
setback. 

To  provide  a  context  in  which  to  evaluate  trends  in 
eliminations  and  setbacks,  data  were  compiled  to  identify  trends 
in  selected  characteristics  of  (a)  the  individuals  who  applied  for 
admission  to  the  Initial  Entry  Rotary  Wing  (lERW)  training  program 
and  (b)  the  individuals  who  were  selected  for  admission  to  the 
lERW  training  program.  The  characteristics  of  interest  included 
FAST  scores,  education  level,  and  prior  military  experience. 

Scope 


Most  analyses  are  based  on  events  that  occurred  during  the  6- 
year  period  between  January  1,  1989,  and  December  31,  1995.  In 
some  instances,  training  events  that  occurred  in  1988  were 
examined  as  well.  Although  the  analysis  of  applicants'  FAST 
scores  was  based  on  the  individuals  who  took  the  FAST  during  the 
period  1989-1995,  some  of  the  individuals  who  participated  in 
training  during  the  period  1989-1995  took  the  FAST  prior  to  1989. 

METHOD 

To  accomplish  the  objectives  described  above,  it  was 
necessary  to  compile  and  tabulate  data  from  a  variety  of  sources. 
Data  on  FAST  scores  were  extracted  from  two  large  databases 
developed  and  maintained  by  the  Army  Research  Institute  (ARI) 
Rotary  Wing  Aviation  Research  Unit  (RWARU) :  the  RFAST  archives 
database  and  the  AFAST  archives  database.  Data  on  the  social 
security  number  (SSN) ,  class  number,  race,  gender,  educational 
level,  and  prior  military  experience  were  obtained  from  one  of  the 
two  databases  described  above  or  from  (a)  the  RWARU  multi-track 
database,  or  (b)  the  Aviation  Center  (USAAVNC)  Automated 
Instructional  Management  System  (AIMS)  database. 

Data  on  eliminations  and  setbacks  came  from  two  sources .  One 
source  of  data  was  briefing  slides  used  by  training  managers  in 
their  annual  review  briefings.  The  second  source  was  the 
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memorandum,  entitled  "Changes  in  Class  Roster, "  that  is  published 
weekly  by  the  Director  of  Plans,  Training,  Mobilization,  and 
Security.  These  memoranda  list  the  type  and  cause  of  each 
training  event  that  resulted  in  a  change  in  the  class  roster, 
including  (but  not  limited  to)  eliminations  and  setbacks.  It  was 
necessary  to  enter  all  data  from  the  Changes  in  Class  Roster  into 
the  computer  manually  in  order  to  create  the  desired  database. 

The  size  and  composition  of  the  databases  developed  for  this 
project  are  shown  in  Appendix  E  (see  Figures  E-1,  E-2 ,  E-3 ,  and  E- 
4)  . 


SUMMARY  OF  FINDINGS  AND  CONCLUSIONS 

A  brief  summary  of  the  findings  is  presented  in  the  following 
subsections.  The  findings  described  in  each  subsection  are 
followed  by  the  conclusions  that  are  supported  by  the  findings. 

Characteristics  of  Applicants 

Appendix  A,  entitled  Characteristics  of  lERW  Applicants, 
contains  6  figures  that  show  the  annual  number  of  applicants  and 
their  FAST^  score,  education  level,  and  prior  military  experience. 

During  the  period  1989-1995,  there  was  a  modest  increase  in 
the  education  level  of  applicants,  as  a  group.  The  percentage  of 
applicants  with  only  a  high  school  education  declined.  There  was 
a  corresponding  increase  in  the  percentage  of  applicants  who  had 
some  college  education  or  had  earned  a  college  degree. 

There  was  a  modest  decrease  in  the  percentage  of  civilian 
applicants  and  a  modest  increase  in  the  percentage  of  applicants 
who  were  ROTC  or  Academy  students.  The  percentages  of  applicants 
who  were  active  duty  enlisted  or  active  duty  officers  remained 
relatively  constant  at  about  50%  and  10%,  respectively. 

For  all  education  levels,  applicants'  FAST  scores  increased 
consistently  over  the  period  1989-1995;  the  increase  varied  from 
about  8  points  to  about  10  points,  depending  on  education  level. 
FAST  scores  were  found  to  be  positively  correlated  with 
applicants '  education  level . 

The  findings  support  the  conclusion  that  the  characteristics 
of  applicants  changed  in  two  ways  during  the  period  1989-1995: 

(a)  applicants'  education  level  has  increased,  and  (b)  FAST  scores 
have  increased  for  all  four  of  the  educational  groups  examined. 


2  The  databases  compiled  for  this  project  included  both  RFAST  scores  and  AFAST 
scores.  For  the  sake  of  simplicity,  the  general  term  "FAST"  is  used 
throughout  this  report  to  refer  to  RFAST  and  AFAST  tests  or  scores  except  in 
the  few  instances  in  which  it  is  important  to  distinguish  between  the  two 
forms  of  the  test. 
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Characteristics  of  lERW  Students 


Appendix  B,  entitled  Characteristics  of  lERW  Students , 
contains  7  figures  that  show  the  annual  number  of  applicants 
selected  for  entry  into  the  lERW  training  program  and  their  FAST 
score,  education  level,  and  prior  military  experience. 

As  was  true  for  applicants,  the  average  education  level  of 
lERW  students  increased  during  the  period  1989-1995:  the 
percentage  of  high  school  graduates  declined,  the  percentage  of 
college  graduates  increased,  and  the  percentage  with  some  college 
remained  about  the  same. 

There  were  no  consistent  changes  in  the  prior  military 
experience  of  lERW  students.  However,  each  year,  between  70%  and 
75%  of  the  students  selected  for  entry  into  the  lERW  training 
program  were  on  active  duty  at  the  time  they  were  selected. 

Although  the  FAST  scores  of  applicants  increased  during  the 
period  1989-1995,  there  was  no  corresponding  change  in  the  FAST 
scores  of  lERW  students.  This  finding  is  surprising.  It  would  be 
expected  that  the  decrease  in  training  slots^  would  have  enabled 
the  Selection  Board  to  select  applicants  with  higher  FAST  scores . 

The  findings  support  the  conclusion  that  the  education  level 
of  lERW  students  has  increased  since  1989  but,  during  the  same 
period,  students'  FAST  scores  remained  about  the  same.  The 
findings  also  support  the  conclusion  that  the  Selection  Board  is 
not  using  FAST  score  as  a  means  to  be  more  selective. 

Eliminations  and  Setbacks 


Appendix  C,  entitled  Niimber  of  Eliminations  and  Setbacks, 
contains  7  figures  that  show  the  annual  number  and  the  causes  of 
eliminations  and  setbacks .  All  of  the  data  in  Appendix  C  were 
obtained  from  briefing  charts  prepared  by  USAAVNC  personnel.  The 
data  identify  only  two  general  causes  of  eliminations  and 
setbacks:  flight  deficiency  and  administrative. 

Flight  deficiency  eliminations  declined  from  a  high  of  177  in 
1991  to  between  9  and  27  in  the  following  years.  Administrative 
eliminations  declined  from  a  high  of  107  in  1990  to  28  in  1995, 
with  the  largest  decline  occurring  between  1990  and  1991. 

Flight  deficiency  setbacks  decreased  from  a  high  of  342  in 
1989  to  a  low  of  74  in  1995.  The  data  for  administrative  setbacks 
was  incomplete.  However,  it  is  known  that  there  were  about  500 
administrative  setbacks  in  both  1990  and  1991  and  that  there  were 
only  129  administrative  setbacks  in  1995. 


^  lERW  training  slots  decreased  from  a  high  of  1,696  in  1991  to  958  in  1995. 
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The  findings  support  the  conclusion  that  both  eliminations 
and  setbacks  have  declined  dramatically  in  recent  years.  The 
USAAVNC  officials  who  were  briefed  on  the  findings  judged  that  the 
data  provide  no  evidence  that  eliminations  and  setbacks  represent 
a  serious  problem  at  the  present  time.  In  particular,  USAAVNC 
officials  judged  that  the  numbers  of  eliminations  and  setbacks  are 
not  great  enough  at  present  to  justify  initiating  work  on  a  new 
aviator  selection  test  battery.  It  is  important  to  acknowledge, 
however,  that  this  conclusion  was  not  based  on  an  objective 
analysis  of  the  relative  cost  of  developing  a  new  selection  test 
batteDTy  and  the  savings  realized  from  reducing  eliminations  and 
setbacks . 

Estimated  Cost  of  Eliminations  and  Setbacks 

Appendix  D,  entitled  Estimated  Cost  of  Eliminations  and 
Setbacks .  contains  9  figures  that  show  the  current  cost  of  each 
phase  of  lERW  training  and  the  cost  of  Aircraft  Qualification 
Course  (AQC)  training  in  each  type  aircraft.  The  figures  also 
show  estimates  of  the  annual  costs  of  eliminations  and  setbacks, 
based  on  the  frequencies  reported  by  USAAVNC . 

Because  elimination  and  setback  costs  are  a  multiple  of  their 
frequencies,  the  cost  trends  are  the  same  as  the  trends,  described 
above,  for  the  number  of  elimination  and  setbacks.  The  cost  data 
quantify  the  enormous  benefits  realized  from  preventing 
eliminations  and  setbacks  through  effective  aviator  selection. 

The  cost  data  also  quantify  the  great  benefits  of  eliminating 
aviators  earlier  rather  than  later  in  the  training  program. 

The  USAAVNC  officials  who  were  briefied  concluded  that  the 
dollar  cost  of  eliminations  and  setbacks  is  not  great  enough  at 
present  to  justify  initiating  work  on  a  new  aviator  selection  test 
battery.  However,  this  conclusion  was  not  based  on  a  careful 
analysis  of  (a)  the  cost  developing  and  validating  a  new  test 
battery  and  (b)  the  savings  associated  with  reduced  eliminations 
and  setbacks  over  the  life  of  a  new  test  battery. 

Databases  Developed 

Appendix  E,  entitled  Size  and  Composition  of  Databases, 
contains  4  figures  that  show  the  size  and  composition  of  the 
databases  developed  to  accomplish  the  objectives  of  this  project. 

The  training  event  database  contains  data  on  virtually  all  of 
the  eliminations  and  setbacks  that  occurred  during  the  period 
1988-1995.  The  database  also  includes  all  events  that  are 
functionally  the  same  as  eliminations  and  setbacks  but  have  a 
different  name.  FAST  scores  were  located  for  about  70%  of  the 
individuals  who  were  eliminated  or  who  experienced  one  or  more 
setbacks . 
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It  was  possible  to  compile  a  large  comparison  database  for 
both  lERW  students  (6,246)  and  AQC  students  (4,372).  However,  the 
sample  size  for  females  and  for  individual  non-white  racial  groups 
was  too  small  to  support  some  of  analyses  that  were  of  interest. 

Analysis  of  Causes  of  Eliminations  and  Setbacks 

Appendix  F,  entitled  Causes  of  Eliminations  and  Setbacks . 
contains  10  tables  and  6  figures  that  show  the  results  of  a 
detailed  analysis  of  the  types  and  frequencies  of  causes  cited  by 
instructors  for  eliminations  and  setbacks.  All  data  for  this 
analysis  came  from  the  weekly  memoranda  listing  changes  in  the 
class  roster,  which  is  published  weekly  by  the  Director  of  Plans, 
Training,  Mobilization,  and  Security.  These  data  were  augmented 
with  information  obtained  during  interviews  with  subject  matter 
experts  (SMEs)  about  the  meaning  of  some  causes  cited  for 
eliminations  and  setbacks. 

The  analysis  identified  274  unique  descriptors  that  were 
cited  as  causes  of  eliminations  or  setbacks.  Some  of  the 
descriptors  were  clear  and  specific  in  their  identification  of  the 
cause  of  the  elimination  or  setback.  However,  a  large  number  of 
descriptors  were  so  vague  that  it  was  difficult  or  impossible  to 
determine  whether  the  cause  was  due  to  (a)  a  skill /knowledge 
deficiency  or  behavioral  problem,  or  (b)  a  personal  problem  or  a 
purely  administrative  problem. 

The  interviews  with  SMEs  revealed  that  many  of  the  causes 
cited  for  eliminations  and  setbacks  do  not  reflect  the  true  cause 
of  the  eliminations  and  setbacks.  The  best  example  is  "medical" 
eliminations  and  setbacks.  The  SMEs  reported  that,  in  many  cases, 
students  who  are  performing  poorly  in  training  claim  that  they 
have  a  medical  problem,  thereby,  receiving  a  medical  elimination 
or  setback  rather  than  a  flight  deficiency  elimination  or  setback. 
The  SMEs  suggested  that  the  use  of  misleading  causes  is  more 
common  during  lERW  training  than  AQC  training. 

The  data  show  that  a  large  number  of  setbacks  are  due  to 
administrative  problems .  It  can  be  argued  that  administrative 
setbacks  replace  some  setbacks  that  result  from  a  knowledge /ski 11 
deficiency  or  from  a  behavior  problem.  That  is,  a  non- 
administrative  setback  may  have  been  necessary  if  a  marginal 
student  had  not  received  the  extra  training  that  was  given  because 
of  an  administrative  setback. 

The  data  in  Appendix  F  support  the  five  general  conclusions 
listed  below. 

•  There  is  a  lack  of  consistency  and  reliability  in  the 
current  practice  of  designating  the  cause  of  eliminations 
and  setbacks . 
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•  Because  of  the  problems  in  designating  cause,  the  number  of 
flight  deficiency  eliminations  and  setbacks  reported  by 
USAAVNC  is  an  underestimate  of  the  number  of  eliminations 
and  setbacks  that  are  the  result  of  a  knowledge /ski 11 
deficiency  or  behavior  problem. 

•  Because  of  the  problems  in  designating  cause,  neither 
eliminations  nor  setbacks  constitute  a  reliable  criterion 
for  use  in  selection  test  validation.  The  large  number  of 
administrative  setbacks  further  reduces  the  value  of 
setbacks  as  a  validation  criterion. 

•  The  number  of  eliminations  and  setbacks  that  are  due  to 
behavior  problems  (e.g.,  misconduct,  honor  code  violation) 
is  large  enough  to  justify  the  development  of  test 
instruments  to  exclude  applicants  with  behavior  problems . 

Interviews  with  SMEs,  all  of  whom  are  highly  experienced 
members  of  the  aviator  training  staff  at  Fort  Rucker,  revealed 
that  the  reported  cause  of  eliminations  and  setbacks  often  does 
not  reflect  the  true  cause.  They  reported  that  the  most  commonly 
misused  causes  are  "resignation,”  "medical,"  and  "personal 
problems . "  The  SMEs  stated  that  many  of  the  eliminations  and 
setbacks  attributed  to  one  of  these  three  causes  are,  in  fact,  due 
to  a  student's  knowledge /ski 11  deficiency  or  behavioral  problem. 
For  example,  the  SMEs  explained  that  a  resignation  is  not  as 
damning  as  a  flight  deficiency  elimination  or  an  elimination  due 
to  a  conduct  problem.  Similarly,  SMEs  explained  that  students  who 
are  doing  poorly  in  training  often  claim  a  medical  problem  and,  as 
a  result,  are  given  a  setback.  The  setback  affords  the  student 
additional  study  time  and,  usually,  additional  flight  hours. 
Similar  examples  were  given  for  the  misuse  of  the  cause  "personal 
problems . " 

The  conclusion  that  the  number  of  flight  deficiency 
eliminations  and  setbacks  is  underestimated  in  the  USAAVNC  reports 
is  based  on  (a)  the  SMEs  reports  of  the  misuse  of  causes  of 
eliminations  and  setbacks  and  (b)  the  relatively  large  number  of 
eliminations  and  setbacks  that  are  attributed  each  year  to. the 
misused  causes:  resignation,  medical,  and  personal  problems. 
Unfortunately,  the  data  do  not  support  quantitative  estimates  of 
the  frequency  with  which  these  causes  are  misused. 

Cut-Score  Analysis 

Appendix  G,  entitled  Results  of  Cut-Score  Analysis,  contains 
10  figures  that  show  the  results  of  the  cut-score  analysis. 

The  results  of  the  cut-score  analysis  were  consistent  across 
training  programs  (lERW  and  AQC) ,  racial  groups  (white  males  vs. 
non-white  males),  and  gender  groups.  For  every  group  and  at  every 
cut-score  between  100  and  120,  more  eliminees  and  individuals  with 
setbacks  would  have  been  excluded  than  individuals  in  the 
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associated  comparison  group.  However,  as  would  be  expected  from 
knowledge  of  the  AFAST's  low  predictive  validity,  the  AFAST  did 
not  prove  to  be  a  highly  effective  discriminator. 

For  white  male  eliminees,  increasing  the  cut-score  to  a  value 
between  100  and  110  would  have  excluded  from  12%  to  18%  more 
eliminees  than  individuals  in  the  comparison  group.  For  non-white 
male  eliminees,  increasing  the  cut-score  to  100  would  have 
excluded  about  22%  more  eliminees  than  individuals  in  the 
comparison  group;  however,  the  benefits  of  increasing  the  cut- 
score  declined  slightly  as  the  cut-score  increased  above  a  value 
of  100.  For  both  white  and  non-white  males,  increasing  cut-score 
would  have  excluded  more  individuals  eliminated  because  of  skill 
or  behavior  problems  than  individuals  who  were  eliminated  for 
other  reasons . 

Increasing  cut-score  was  found  to  be  less  effective  for 
excluding  individuals  with  setbacks  than  excluding  eliminees.  In 
most  instances,  increasing  the  cut-score  to  any  value  between  100 
and  120  would  have  excluded  about  5-8%  more  individuals  with 
setbacks  than  individuals  in  the  comparison  group.  For  white 
males,  increasing  cut-score  would  have  excluded  5-12%  more 
individuals  with  setbacks  due  to  skill  or  behavior  problems  than 
individuals  with  setbacks  due  to  other  problems.  However,  this 
difference  was  not  as  large  and  not  as  consistent  for  other 
groups . 

The  data  in  Appendix  G  support  the  conclusion  that  limited 
benefits  would  be  realized  from  increasing  the  FAST  score  from  90 
(current  cut-score)  to  a  value  between  100  and  120.  That  is, 
increasing  cut-score  would  result  in  the  exclusion  of 
proportionately  fewer  successful  students  (students  who  complete 
training  with  no  setbacks)  than  unsuccessful  students  (students 
who  could  not  complete  training  or  who  required  one  or  more 
setbacks) .  However,  the  feasibility  of  increasing  cut-score 
depends  on  whether  the  applicant  pool  is  large  enough  to  fill  the 
training  slots  with  individuals  with  a  FAST  score  as  high  or 
higher  than  the  new  cut-score. 

RECOMMENDATIONS 

It  is  recommended  that  USAAVNC  develop  and  implement  an 
improved  system  for  designating  the  true  cause  of  eliminations  and 
setbacks .  It  is  further  recommended  that  a  computer-based  system 
be  developed  that  requires  instructors  to  select  the  appropriate 
cause  from  among  a  prescribed  list  of  causes  and  that  instructor 
personnel  be  required  to  enter  detailed  information  about  a 
student ' s  performance  at  the  time  of  a  medical  or  any  other  type 
of  administrative  elimination  or  setback.  The  resulting  data 
would  be  extremely  valuable  for  training  managers;  the  data  are 
essential  for  validating  current  and  future  aviator  selection 
tests. 
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Because  increasing  the  cut-score  is  mainly  an  administrative 
decision,  it  is  recommended  that  Selection  Board  representatives 
examine  the  feasibility  and  acceptability  of  increasing  the  cut- 
score  from  90  to  a  value  between  100  and  110.  This  examination 
should  consider  both  the  size  and  qualifications  of  the  applicant 
pool  and  the  likelihood  that  a  change  in  cut-score  would  stimulate 
claims  of  race  or  gender  bias. 

A  final  recommendation  is  to  develop  an  improved  system  to 
assess  aviators '  proficiency  and  effectiveness  upon  completion  of 
institutional  training  and  assignment  to  an  operational  unit.  The 
importance  of  this  recommendation  cannot  be  overemphasized.  The 
recommendation  stems  from  two  related  concerns .  One  concern  is 
the  declining  predictive  validity  of  the  FAST.  The  second  concern 
is  the  continued  decline  in  training  resources  and  the  resultant 
increase  in  the  pressure  on  members  of  the  training  community  to 
minimize  the  losses  associated  with  eliminations  and  setbacks.  As 
the  quality  of  candidates  continue  to  decline,  a  time  must  come 
when  it  is  simply  not  possible  to  maintain  elimination  and  setback 
rates  at  their  current  low  level  without  a  decline  in  the 
proficiency  and  effectiveness  of  the  average  graduate.  To  avoid 
the  unacceptable  consequences  of  a  decline  in  graduate 
proficiency,  it  will  be  necessary  to  create  a  performance 
assessment  system  that  is  independent  of  the  institutional 
training  community  and  the  pressures  to  keep  the  elimination  and 
setback  rates  low.  Because  of  the  need  to  detect  proficiency 
changes  over  time,  it  is  essential  that  the  system  employ 
proficiency  assessment  methods  and  procedures  that  ensure  that 
both  performance  standards  and  perfoDonance  measures  remain  stable 
over  time. 

Although  the  cost  of  developing  and  operating  a  post  training 
proficiency  assessment  system  is  certain  to  be  relatively  high, 
the  resulting  proficiency  measurement  data  will  benefit  the  Army 
in  two  important  ways.  First,  the  data  will  enable  Army  officials 
to  monitor  the  proficiency  of  their  graduate  aviators  and  to 
provide  immediate  feedback  to  the  institutional  training  community 
in  the  event  that  a  decline  in  proficiency  is  detected.  Secondly, 
the  performance  measures  will  provide  a  highly  useful  validation 
criterion  for  use  in  validating  current  and  future  aviator 
selection  tests . 

Although  USAAVNC  officials  concluded  that  the  current  number 
of  eliminations  and  setbacks  is  not  large  enough  to  justify  a 
recommendation  to  commence  work  on  a  new  test  battery  to  replace 
the  aging  AFAST,  a  number  of  events  could  result  in  an  immediate 
and  pressing  need  for  a  new  aviator  selection  test  battery.  One 
such  event  is  a  serious  compromise  of  the  AFAST.  Another  is  a 
military  crisis  that  results  in  a  sudden  increase  in  the 
requirement  for  trained  Army  aviators. 

If  such  events  occur,  it  can  be  expected  that  at  least  a  3- 
year  effort  would  be  required  to  develop  and  validate  a  new 
aviator  selection  test  battery.  Test  development  could  be 
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expedited  somewhat  by  increasing  the  number  and  expertise  of  the 
personnel  assigned  to  a  test  development  team.  However,  little 
can  be  done  to  expedite  the  validation  process.  The  effectiveness 
of  test  validation  is  largely  dependent  on  the  number  of  students 
who  have  taken  the  test  and  whose  perfo2nnance  in  training  is 
known.  Hence,  the  elapsed  time  required  to  validate  a  selection 
test  battery  is  mainly  a  function  of  the  size  of  the  student 
population. 
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Appendix  A 

Characteristics  of  lERW  Applicants 


Comments : 

The  figures  in  Appendix  A  show  selected  characteristics  of 
all  individuals  who  applied  for  entry  into  the  lERW  program 
during  the  period  1989-1995.  The  characteristics  examined 
include  education  level,  prior  military  experience,  and  FAST 
score.  It  should  be  noted  that  any  individual  who  took  the 
FAST  test  was  considered  an  applicant  even  though  it  is  known 
that  some  ROTC  students  and  some  Army  Academy  students  who 
took  the  test  had  no  serious  intent  to  apply  for  entry  into 
the  flight  training  program.  To  illustrate  changes  over 
time . 


A-1 


Year 

*GED  =  General  Education  Diploma 


Figure  A-1.  Education  level  of  individuals  who  took  the 
FAST  during  the  period  between  1989  and  1995. 


Comments : 

The  individuals  who  take  the  FAST  each  year  are 
considered  the  applicant  pool.  However,  not  all  the 
individuals  in  the  applicant  pool  apply  for  admission  to 
flight  school  after  taking  the  test. 

Figure  A-1  shows  that  the  average  level  of  individuals 
in  the  applicant  pool  increased  during  the  five-year 
period  shown.  High  school  graduates  decreased  by  about 
17%  while  college  graduates  increased  by  about  9%  and 
individuals  with  some  college  increased  by  about  12%. 

The  percentage  of  individuals  with  only  a  GED  remained 
uniformly  low. 
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Figure  A-2 .  Number  of  FAST  tests  administered  per  year 
by  education  level. 


Comments : 

Figure  A-2  shows  the  large  decrease  in  the  nimber  of 
FAST  tests  administered  per  year  during  the  period 
between  1989  and  1995.  Although  some  decrease  is  shown 
for  all  four  education  levels,  the  largest  decrease  is 
the  number  of  high  school  graduates  and  the  number  of 
individuals  with  some  college.  In  1995,  about  the  same 
niomber  of  college  graduates  were  tested  as  high  school 
graduates . 

The  available  data  show  that  the  total  number  of  FAST 
tests  administered  decreased  from  6,570  in  1989  to  2,407 
in  1995.  However,  the  ARI  database  does  not  include 
data  on  every  FAST  test  that  is  administered. 
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Figure  A-3 .  Prior  military  experience  of  individuals 
who  took  the  FAST  during  the  period  between  1989  and 
1995. 


Comments : 


Active  duty  enlisted  accounted  for  more  than  50%  of  the 
individuals  who  took  the  FAST,  and  active  duty  officers 
accounted  for  fewer  than  10%.  The  percent  who  were 
civilians  at  the  time  they  took  the  test  declined  from 
about  25%  to  about  15%.  The  percentage  who  were  ROTC  or 
Academy  students  or  recent  graduates  at  the  time  they 
took  the  test  increased  from  about  14%  to  about  26%. 


Year 

*GED  =  General  Education  Diploma 

Figure  A-4.  Applicants'  FAST  scores  as  a  function  of 
education  level  and  year. 


Coininents : 

The  FAST  scores  of  applicants  are  positively  related  to 
the  applicants'  education  level.  The  test  scores  of 
college  graduates  is  no  more  than  5  points  higher  than 
the  test  scores  of  individuals  with  some  college,  but  is 
nearly  10  points  higher  than  the  test  scores  of  high 
school  graduates.  (It  should  be  noted  that  some 
individuals  with  some  college  at  the  time  they  took  the 
test  subsequently  completed  college.)  Although  there  is 
no  apparent  difference  in  the  FAST  scores  of  high  school 
graduates  and  individuals  with  a  GED,  the  number  of 
individuals  with  only  a  GED  is  very  small. 

For  all  education  levels,  FAST  scores  increased 
consistently  as  a  function  of  time.  Average  FAST  scores 
increased  by  a  value  that  varied  from  about  8  points 
(high  school  graduates)  to  about  10  points  (college 
graduates) . 
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Figure  A-5.  Mean  and  standard  deviation  of  applicants' 
FAST  scores  as  a  function  of  education  level  at  the  time 
the  FAST  was  taken. 


Comment : 

The  mean  FAST  scores  of  applicants  are  positively 
related  to  applicants'  education  level,  but  the 
difference  is  not  large.  The  mean  score  for  college 
graduates  (111.2)  is  only  about  one-half  standard 
deviation  larger  than  the  mean  score  of  individuals  with 
only  a  GED  (99.7) .  As  can  be  seen,  the  standard 
deviation  is  nearly  the  same  for  all  educational  groups; 
the  difference  between  the  smallest  and  largest  standard 
deviation  is  less  than  one  point. 
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Appendix  B 

Characteristics  of  lERW  Students 


Coiranents : 

The  figures  in  Appendix  B  show  selected  characteristics  of 
individuals  who  participated  as  students  in  the  lERW  training 
program  during  the  period  1989-1995.  The  characteristics 
examined  include  education  level,  prior  military  experience, 
and  FAST  score.  It  should  be  noted  that  any  individual  who 
took  the  FAST  test  was  considered  an  applicant  even  though  it 
is  known  that  some  ROTC  students  and  some  Army  Academy 
students  who  took  the  FAST  had  no  serious  intent  to  apply  for 
entry  into  the  flight  training  program.  To  illustrate 
changes  over  time  the  data  are  shown  by  year  for  the  7 -year 
period  between  1989  and  1995. 
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Figure  B-1.  Education  level  of  lERW  flight  students  as 
a  function  of  year  of  entry. 


Comment : 

Between  1989  and  1995,  college  graduates  accounted  for  a 
progressively  larger  percentage  of  the  applicants  who 
were  selected  for  entry  into  the  lERW  training  program. 
The  percentage  of  college  graduates  increased  from  about 
43%  in  1989  to  about  63%  in  1995.  At  the  same  time, 
high  school  graduates  accounted  for  a  progressively 
smaller  percentage  of  the  applicants  selected;  the 
percentage  decreased  from  about  20%  in  1989  to  about  4% 
in  1995.  The  percentage  of  applicants  selected  with 
some  college  remained  about  the  same. 

Comparison  of  the  above  data  with  the  data  in  Figure  A-1 
shows  that  although  only  about  22%  of  1995  applicants 
were  college  graduates,  about  63%  of  the  applicants 
selected  that  year  were  college  graduates . 


B-2 


Year 

*AD  =  Active  E>uty 


Figure  B-2 .  Prior  military  experience  of  individuals 
who  were  selected  for  entry  into  lERW  training  during 
the  period  between  1989  and  1995. 


Comments : 

Most  of  the  applicants  selected  for  entry  into  lERW  were 
on  active  duty  at  the  time  they  were  selected.  Although 
the  trends  over  time  are  not  consistent,  the  relative  - 
percentage  of  active  duty  enlisted  tended  to  decrease, 
and  the  relative  percentage  of  active  duty  officers 
tended  to  increase  during  the  period  between  1989  and 
1995. 

The  relative  percentage  of  civilians  selected  decreased 
from  about  15%  in  1989  to  about  6%  in  1995.  ROTC  or 
Academy  graduates  accounted  for  between  15%  and  23%  of 
the  applicants  selected,  depending  on  the  year. 
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FAST  Score 


Figure  B-3 .  Gentiles  of  FAST  scores  for  individuals  who 
were  selected  for  entry  into  the  lERW  training  program 
during  the  period  between  1989  and  1995. 


Comments : 

There  has  been  little  change  in  the  distribution  of  FAST 
scores  for  the  individuals  selected  for  entry  into  lERW 
between  1989  and  1995 .  The  only  noteworthy  change  is 
the  decrease  in  the  5th  centile  scores  for  1994  and 
1995.  In  1994,  the  5th  centile  was  below  the  cut  score 
of  90;  the  5th  centile  was  the  same  as  the  cut  score  in 
1995.  This  indicates  that  5%  or  more  of  the  applicants 
selected  in  1994  and  1995  had  FAST  scores  below  the  cut 
score.  Similarly,  the  25th  centile  scores  were  lower  in 
1994  and  1995  than  in  the  two  prior  years. 


B-4 


Median  FAST  Score 


150 

140 

130 

120 

110 

100 

90 

80 

Figure  B-4 .  Median  FAST  scores  for  all  applicants  and 
for  -the  applicants  who  were  selected  for  entry  into  the 
lERW  training  program  during  the  period  between  1989  and 
1995. 


Comments : 


Between  1989  and  1995,  the  median  FAST  score  of 
applicants  increased  consistently  from  about  101  to 
about  111 .  There  was  no  corresponding  increase  in  the 
FAST  score  of  the  applicants  that  were  selected. 
Although  the  median  FAST  score  of  the  applicants 
selected  remained  higher  than  the  median  score  of  the 
total  applicant  pool,  this  difference  had  decreased  to 
less  than  10%  by  1995. 
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Figure  B-5.  Distribution  of  individuals  tested  and 
individuals  selected  for  entry  into  the  lERW  training 
program  as  a  function  of  prior  military  service  at  the 
time  of  testing. 


Comments : 

The  distribution  of  individuals  who  were  tested  (FAST) 
is  similar  to  the  distribution  of  the  individuals 
selected  for  entry  into  the  lERW  training  program.  Only 
two  differences  are  noteworthy.  First,  proportionately 
more  (about  8%)  active  duty  officers  were  selected  than 
were  tested  and,  second,  proportionately  fewer  (about’ 

6%)  civilians  were  selected  than  were  tested.  It  should 
be  noted  that  many  ROTC  and  Academy  students  take  the 
FAST  even  though  they  have  no  serious  intention  to  apply 
for  admission  to  the  flight  training  program.  For  this 
reason,  it  is  probable  that  proportionately  more  ROTC 
and  Academy  students  are  selected  than  actually  apply 
for  admission  to  flight  training. 
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Figure  B-6.  lERW  training  program  applicants  selected 
by  year  during  the  period  between  1989  and  1995. 


Comments : 

The  number  of  applicants  selected  for  entry  into  the 
lERW  training  program  declined  from  a  high  of  1,696  in 
1991  to  a  low  of  958  in  1995.  The  number  declined 
further,  to  778,  in  1996. 
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Appendix  C 

Number  of  Eliminations  and  Setbacks 


Comments : 

The  figures  in  Appendix  C  show  the  number  of  individuals 
eliminated  from  flight  training  and  the  number  of  instances 
in  which  an  individual  was  set  back  during  training.  The 
data  shown  in  Appendix  C  tables  are  data  that  were  reported 
by  USAAVNC  training  officials  during  annual  briefings.  That 
is,  all  the  figures  in  this  appendix  are  based  on  data 
compiled  from  briefing  charts  that  were  retrieved  from  the 
USAAVNC  files .  The  frequencies  represent  the  combined  number 
(eliminations  or  setbacks)  that  occurred  during  lERW  training 
and  during  AQC  training.  All  tables  show  frequencies  by  year 
for  the  7-year  period  between  1989  and  1995. 
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Figure  C-1.  Yearly  eliminations  and  setbacks  reported 
by  the  USAAVNC  during  the  period  between  1989  and  1995. 


Comments : 

The  data  presented  above  include  all  eliminations  and 
setbacks,  regardless  of  cause.  Some  of  the  eliminations 
and  setbacks  are  due  to  flight  deficiencies  and  others 
are  due  to  administrative  problems . 

The  reasons  for  the  dramatic  decrease  in  setbacks  in 
1992  is  not  known  for  certain.  It  has  been  suggested 
that  this  decrease  in  setbacks  was  due  to  the  temporairy 
introduction  of  a  one-setback  rule,  which  stipulates 
that  a  flight  student  is  permitted  only  one  flight 
deficiency  setback  before  elimination.  If  this  was  the 
only  factor  in  operation,  it  would  be  expected  that 
eliminations  would  increase  in  number  as  setbacks 
decreased  in  number.  This  was  not  the  case. 
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Figure  C-2 ,  Eliminations  due  to  flight  deficiencies  as 
a  function  of  year  and  training  phase  in  which 
elimination  occurred.  Based  on  data  reported  by 
USAAVNC . 


Comments : 

The  number  of  flight  eliminations  was  far  larger  in  1990 
(82)  and  1991(177)  than  in  all  other  years(27  or  fewer). 
It  has  been  suggested  that  the  large  number  of  flight 
deficiency  eliminations  in  1990  and  1991  was  due  to  the 
temporary  adoption  of  the  one  setback  rule  (each  student 
is  permitted  only  one  flight  deficiency  setback  before 
elimination) .  However,  it  is  probable  that  the 
requirement  to  downsize  the  Army  aviator  population  also 
contributed  to  the  large  number  of  flight  deficiency 
eliminations  in  1990  and  1991. 

It  is  interesting  to  note  that  the  dramatic  changes  in 
flight  deficiency  eliminations  in  1990  and  1991  were  not 
accompanied  by  large  changes  in  student  input. 
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Figure  C-3 .  Eliminations  due  to  administrative  problems 
as  a  function  of  year  and  training  phase  in  which 
eliminations  occurred.  Based  on  data  reported  by 
USAAVNC . 


Comments : 

The  number  of  administrative  eliminations  was  far  higher 
in  1990  (107)  than  in  any  other  year.  It  is  interesting 
to  note  that,  in  1991,  the  number  of  administrative 
eliminations  declined  at  the  same  time  that  the  number 
of  flight  deficiency  eliminations  increased  to  its 
highest  level . 

For  the  4  years  following  1991,  the  number  of 
administrative  eliminations  was  positively  related  to 
student  input. 
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Figure  C-4.  Setbacks  due  to  flight  deficiencies  as  a 
function  of  year  and  training  phase  in  which  setbacks 
occurred.  Based  on  data  reported  by  USAAVNC. 


Comments : 

Flight  deficiencies  were  far  higher  (342)  in  1989  than 
in  any  following  year  (202  or  fewer) .  The  large  number 
of  flight  deficiency  setbacks  in  1989  may  have  led  to 
the  temporary  adoption  of  the  one  setback  rule  in  1990. 
These  findings  (increased  flight  deficiency  eliminations 
and  decrease  flight  deficiency  setbacks  in  1990  and 
1991)  are  the  results  that  would  be  expected  from  the 
introduction  of  a  one  setback  rule. 

It  is  noteworthy  that  the  number  of  flight  deficiency 
setbacks  is  not  closely  related  to  student  input. 
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Figure  C-5.  Setbacks  due  to  administrative  problems  as 
a  function  of  year.  Based  on  data  reported  by  USAAVNC. 
Valid  data  for  1992  and  1993  are  not  available. 


Comments : 

The  data  reported  by  the  USAA.VNC  did  not  contain  the 
number  of  administrative  setbacks  for  1992.  Moreover, 
the  number  of  administrative  setbacks  reported  for  1993 
are  suspect.  For  these  reasons,  it  is  not  possible  to 
comment  about  trends  in  administrative  setbacks  using 
the  data  reported  by  the  USAA.VNC.  It  can  only  be  said 
that  the  number  of  administrative  setbacks  is  at  least 
as  large  as  is  shown  in  Figure  C-5. 
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Figure  C-6.  Eliminations  as  a  function  of  year  and 
cause  (flight  deficiency  or  administrative  problem) . 
Based  on  data  reported  by  USAAVNC. 


Comments : 

Figure  C-6  shows  the  relationship  between  flight 
deficiency  eliminations  and  administrative  eliminations 
during  the  period  between  1989  and  1995.  Except  in  1991 
and  1992,  the  proportion  of  administrative  eliminations 
was  far  higher  than  the  proportion  of  flight  deficiency 
eliminations.  The  large  absolute  and  relative  niimber  of 
administrative  eliminations  is  surprising.  It  was  this 
finding  that  led  to  a  more  comprehensive  study  of  the 
causes  of  administrative  eliminations.  The  results  of 
the  study  of  causes  of  eliminations  and  setbacks  are 
presented  elsewhere  in  this  report. 
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Appendix  D 

Estimated  Cost  of  Eliminations  and  Setbacks 


Comments : 

The  first  two  figures  in  this  appendix  (D-l  and  D-2)  show  the 
estimated  cost  of  each  phase  of  lERW  training  and  the 
estimated  cost  of  each  AQC  training  program.  These  cost 
estimates  were  derived  by  the  USAAVNC  Directorate  of  Resource 
Management  (DRM)  using  1996  cost  data. 

The  remaining  figures  in  Appendix  D  show  estimates  of  the 
annual  dollar  cost  of  eliminations  and  setbacks.  Cost  data 
are  shown  for  flight  deficiency  eliminations  and  setbacks  and 
for  administrative  eliminations  and  setbacks . 
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Figure  D-2  .  Estimated  cost  of  Army  aviator  training  for 
the  single-track  program.  Cost  included  pre-flight  training, 
lERW  training,  and  AQC  training.  Cost  estimates  were  calculated 
by  the  USAAVNC  DRM. 


Comments : 

The  above  data  were  used  to  estimate  the  dollar  costs  of 
eliminations  and  setbacks.  These  data  show  the  enormous 
costs  of  eliminations,  particularly  those  that  occur  late  in 
training. 


D-3 


Figure  D-3 .  Estimated  cost  of  flight  deficiency  eliminations  by 
year  and  training  phase  in  which  elimination  occurred.  Based  on 
elimination  data  and  cost  data  reported  by  USAAVNC. 


Comments : 

The  cost  data  shown  in  Figures  D-1  and  D-2  and  the  elimination 
data  shown  in  Figure  C-2  were  used  to  estimate  the  dollar  cost  of 
flight  deficiency  eliminations.  The  cost  of  an  elimination  was 
estimated  by  adding  one-half  the  cost  of  the  training  phase  in 
which  the  student  was  eliminated  to  the  full  cost  of  all  training 
phases  completed  successfully.  This  estimate  is  conservative  for 
three  reasons.  First,  the  estimated  costs  do  not  include  the  cost 
of  the  setbacks  that  occurred  prior  to  elimination.  Second,  it  is 
probable  that  the  mean  cost  exceeds  one-half  of  the  cost  of  the 
training  phase  in  which  the  student  was  eliminated.  Third,  as  is 
discussed  later,  there  is  compelling  evidence  that  the  true  number 
of  flight  deficiency  eliminations  exceeds  the  number  reported  by 
USAAVNC . 

These  data  clearly  illustrate  the  enormous  dollar  costs  of 
eliminations  in  1990  (about  $10  million  dollars)  and  1991  (about 
$22  million) .  These  data  also  show  the  enormous  savings  that 
would  be  realized  if  flight  deficiency  eliminees  could  be 
identified  during  the  core  phase  of  training  rather  than  later  in 
the  track  phase  of  training. 
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Figure  D-4.  Estimated  cost  of  administrative  eliminations  by- 
year  and  training  phase  in  which  eliminations  occurred.  Based 
on  elimination  data  and  cost  data  reported  by  USAAVNC. 


Comments : 

The  cost  data  shown  in  Figures  D-1  and  D-2  and  the  elimination 
data  shown  in  Figure  C-3  were  used  to  estimate  the  dollar  cost 
of  flight  deficiency  eliminations.  The  cost  of  an  elimination 
was  estimated  by  adding  one-half  the  cost  of  the  training  phase 
in  which  the  student  was  eliminated  to  the  full  cost  of  all 
training  phases  completed  successfully.  This  estimate  is 
conservative  for  two  reasons.  First,  the  estimated  costs  do  not 
include  the  cost  of  the  setbacks  that  occurred  prior  to 
elimination.  Second,  it  is  probable  that  the  mean  cost  exceeds 
one-half  of  the  cost  of  the  training  phase  in  which  the  student 
was  eliminated.  However,  the  cost  of  administrative 
eliminations  is  overestimated  to  the  extent  that  a  flight 
deficiency  was  the  true  cause  of  an  administrative  elimination. 

After  the  peak  in  1990  (nearly  $15  million) ,  the  cost  of 
administrative  eliminations  declined  to  a  low  of  about  $3.5 
million  in  1995. 
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Figure  D-5.  Estimated  cost  of  all  eliminations  by  year 
and  training  phase  in  which  eliminations  occurred. 

Based  on  elimination  data  and  cost  data  reported  by 
USAAVNC . 


Comments : 

The  cost  data  shown  in  Figure  D-5  represent  the  combined 
cost  of  flight  deficiency  eliminations  (Figure  D-3 )  and 
administrative  eliminations  (Figure  D-4) . 
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Figure  D-6.  Estimated  cost  of  flight  deficiency  setbacks 
by  year  and  training  phase  in  which  the  setbacks  occurred. 
Based  on  elimination  data  and  cost  data  reported  by 
USAAVNC . 


Comments : 

The  costs  presented  above  are  based  on  an  estimated  average 
cost  of  $20,000  per  setback.  The  average  cost  per  setback 
was  derived  from  a  study  of  the  number  of  additional  hours 
of  flight  hours  received  per  setback  and  the  associated 
cost  of  student  remuneration,  instructor  pilot 
remuneration,  and  aircraft  costs.  The  cost  estimates  do 
not  include  the  additional  administrative  costs  associated 
with  setbacks . 

In  1989,  the  annual  cost  of  flight  deficiency  setbacks 
($6.8  million)  exceeded  the  cost  of  flight  deficiency 
eliminations  ($2.8  million).  For  all  other  years, 
however, the  annual  cost  of  flight  deficiency  setbacks  is 
far  less  than  the  annual  cost  of  flight  deficiency 
eliminations . 
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Figure  D-7 .  Estimated  cost  of  setbacks  due  to  administrative 
problems  as  a  function  of  year  and  training  phase  during  which  the 
setbacks  occurred.  Based  on  setback  data  and  cost  data  reported 
by  USAAVNC.  Valid  data  for  1992  and  1993  are  not  available. 


Comments : 

The  data  reported  by  the  USAAVNC  did  not  contain  the  number  of 
administrative  setbacks  for  1992.  Moreover,  the  number  of 
administrative  setbacks  reported  for  1993  are  suspect.  For  these 
reasons,  it  is  not  possible  to  comment  about  trends  in  the  number 
or  cost  of  administrative  setbacks  using  the  data  reported  by  the 
USAAVNC.  It  can  only  be  said  that  the  cost  of  administrative 
setbacks  is  at  least  as  large  as  is  shown  in  Figure  C-5 . 

For  the  years  in  which  valid  data  are  available  (all  years  except 
1992  and  1993),  the  cost  of  track  setbacks  is  far  larger  than  the 
cost  of  core  setbacks.  Because  the  estimated  cost  of  all  setbacks 
is  the  same,  this  difference  is  due  solely  to  the  smaller  number 
of  core  setbacks . 
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Figure  D-8.  Estimated  cost  of  flight  deficiency- 
eliminations  and  setbacks  by  year.  Based  on  frequency 
and  cost  data  reported  by  USAAVNC. 


Comments : 


Figure  D-8  shows  the  combined  costs  of  flight  deficiency 
eliminations  and  flight  deficiency  setbacks.  It  is 
noteworthy  that  the  relative  cost  of  flight  deficiency 
eliminations  and  flight  deficiency  setbacks  varies 
greatly  from  year  to  year.  The  relative  cost  of 
eliminations  was  higher  than  setbacks  in  1990,  1991,  and 
1995  but  was  lower  than  setbacks  in  1989,  1992,  1993, 
and  1994. 

Although  the  combined  cost  of  flight  deficiency 
eliminations  and  setbacks  was  very  high  in  1991  ($25.9 
million),  the  combined  cost  was  only  $3.4  million  and 
$4.1  million  in  1994  and  1995,  respectively. 
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Figure  D-9 .  Estimated  cost  of  all  eliminations  and  all 
setbacks  by  year.  Based  on  frequency  and  cost  data 
provided  by  USAAVNC.  Valid  data  for  administrative 
setbacks  are  not  available  for  1991  and  1992 . 


Comments : 

The  estimated  cost  of  all  eliminations  and  setbacks 
increased  from  $22.6  million  in  1989  to  $42.8  million  in 
1991.  Thereafter,  the  cost  decreased  dramatically  to  a 
value  between  $15.4  million  (1994)  and  $10.4  million 
(1995)  .  However,  as  stated  earlier,  the  estimated  cost 
of  setbacks  is  severely  underestimated  for  1992  and  1993 
because  of  the  lack  of  valid  data  on  the  number  of 
administrative  setbacks  that  occurred  during  1992  and 
1993  . 
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Appendix  E 

Size  and  Composition  of  Databases 


Comments : 

The  four  figures  in  Appendix  E  show  the  size  and  composition 
of  the  databases  created  for  use  in  analyzing  (a)  the  types 
and  frequencies  of  the  causes  cited  for  eliminations  and 
setbacks,  and  (b)  the  analysis  of  the  effect  of  increasing 
the  FAST  cut-score.  The  databases  were  compiled  from  the 
weekly  training  events  reports  distributed  weekly  by  the 
UST^VNC . 
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Figure  E-1.  Illustration  of  the  size  and  composition  of 
the  Training  Event  Database. 


Comments : 

The  training  went  database  included  1,384  training  events 
that  occurred  in  training  courses  other  than  the  lERW 
training  or  AQC  training.  Examples  of  such  courses  include 
the  instructor  pilot  course,  the  maintenance  test  pilot 
course,  and  the  instrument  flight  examiner  course.  Training 
events  that  occurred  during  training  courses  other  than  lERW 
and  AQC  were  not  analyzed. 
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Figure  E-2 .  Illustration  of  the  size  and  composition  of 
the  lERW  training  event  database. 


Comments : 

The  lERW  database  contained  891  eliminations  and  4,672 
setbacks.  There  were  877  persons  who  were  eliminated  and 
3,515  who  had  one  or  more  setbacks.  This  means  that  13 
persons  were  found  who  were  eliminated  twice  and  that 
numerous  persons  had  2  or  more  setbacks.  FAST  scores  were 
located  for  67%  of  the  eliminees  and  71%  of  the  individuals 
with  one  or  more  setbacks.  FOrty-two  percent  of  the 
eliminees  were  eliminated  because  of  a  skill  deficiency  or 
a  behavior  problem  (improper  conduct)  ;  the  remaining  5  9%  were 
eliminated  because  of  other  problems  (medical,  IP  shortage, 
etc.).  Ihirty-two  percent  of  the  setbacks  were  due  to 
skill  or  behavior  problems;  the  remaining  68%  were  due  to 
other  problems.  Gender  was  unknown  for  34%  of  the  eliminees 
and  38%  of  the  individuals  with  one  or  more  setbacks.  The 
number  of  known  females  was  small  for  all  four  subsamples. 
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Figure  E-3 .  Illustration  of  the  size  and  composition  of 
the  AQC  training  event  database. 


Comments : 

The  AQC  database  contained  210  eliminations  and  581 
setbacks.  There  were  207  persons  who  were  eliminated  and 
515  who.  had  one  or  more  setbacks.  This  means  that  3 
persons  were  found  who  were  eliminated  twice  and  that 
many  persons  had  2  or  more  setbacks.  FAST  scores  were 
located  for  52%  of  the  eliminees  and  66%  of  the  individuals 
with  one  or  more  setbacks  .  TA?enty-one  percent  of  the 
eliminees  were  eliminated  because  of  a  skill  deficiency  or 
a  behavior  problem  (improper  conduct);  the  remaining  79% 
were  eliminated  because  of  other  problems  (medical,  IP 
shortage,  etc.).  Fifty-six  percent  of  the  setbacks  were  due 
to  skill  or  behavior  problems;  the  remaining  44%  were  due  to 
other  problems.  Gender  was  unknown  for  46%  of  the  eliminees 
and  38%  of  the  individuals  with  one  or  more  setbacks.  The 
number  of  known  females  was  small  for  all  four  subsamples. 
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COMPARISON  DATABASE 


_ t _ 

lERW  COMPARISON 

•  Total  =  6,24  6 

•  Males  =  5,98  0 

•  Females  =264 

•  White  =  3,906  Males 

•  Black  =  161  Males 

•  Hispanic  =  141  Males 

•  Asian  =39  Males 

•  Indian*  =  29  Males 


_ f _ 

AQC  COMPARISON 

•  Total  =  4,37  2 

•  Males  =  4,209 

•  Females  =159 

•  White  =  3,749  Males 

•  Black  =  18  6  Males 

•  Hispanic  =  158  Males 

•  Asian  =  79  Males 

•  Indian*  =  17  Males 


*American  Indian 


Figure  E-4.  Illustration  of  the  size  and  composition  of 
the  comparison  database. 


Comments : 

The  comparison  database  includes  individuals  who  completed 
lERW  or  AQC  training  during  the  period  1988-1995  and  met  each 
of  the  following  criteria:  (a)  experienced  no  eliminations  or 
setbacks,  and  (b)  had  a  valid  FAST  test  score  on  file  at  ARI. 
Although  both  the  lERW  comparison  database  and  the  AQC 
comparison  database  are  large,  white  males  accounted 
for  63%  of  the  individuals  in  the  lERW  comparison  group  and 
8  6%  of  the  individuals  in  the  AQC  comparison  group.  No  other 
single  racial  or  gender  group  accounts  for  more  than  4.4%  to  the 
total . 
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Appendix  F 

Causes  of  Eliminations  and  Setbacks 


Comments : 

In  the  following  tables  and  figures,  the  term  "eliminations" 
refers  to  training  events  in  which  a  student  "resigned, "  was 
"eliminated, "  or  was  "relieved"  and  not  reinstated. 

Similarly,  the  term  "setbacks"  includes  training  events  in 
which  a  student  was  "setback,"  "turned  back,"  or  was 
"relieved  and  reinstated. " 

The  database  contained  274  unique  descriptors  that  were  cited 
as  causes  of  eliminations,  setbacks,  or  both.  Some 
descriptors  were  clear  and  specific  in  their  identification 
of  the  cause  of  the  elimination  or  setback.  However,  a  large 
number  of  descriptors  were  so  vague  that  it  was  difficult  or 
impossible  to  determine  whether  the  cause  was  due  to  (a)  a 
skill /knowledge  deficiency  or  behavioral  problem,  or  (b)  a 
personal  problem  or  a  purely  administrative  problem.  In 
spite  of  this  difficulty,  all  descriptors  of  cause  were 
classified  into  two  broad  categories  and  numerous  sub¬ 
categories  .  One  broad  category  included  descriptors  that 
implied  a  knowledge /ski 11  deficiency  or  a  behavioral  problem; 
the  other  category  included  all  other  descriptors. 

Tables  F-1  and  F-2  list  the  causes  of  eliminations  and 
setbacks  that  occurred  prior  to  the  time  the  student 
commenced  flight  training.  The  remaining  tables  (F-3  through 
F-10)  list  the  causes  of  eliminations  and  setbacks  that 
occurred  during  flight  training.  Separate  tables  are  shown 
for  lERW  and  AQC  training. 
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Table  F-1 


Causes  cited  for  eliminations  prior  to  the  start  of  lERW 
flight  training. 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

%  GROUP* 
(N=232) 

%  TOTAL** 
(N=891) 

Resignation 

101 

43.5 

11.3 

Medical /Fitness 

43 

18.5 

4.8 

Misconduct 

24 

10.3 

2.7 

Failure  to  Progress 

20 

8.6 

2.2 

Unit  Recall 

15 

6.5 

1.7 

Other  Administrative 

7 

3.0 

0.8 

Academic 

5 

2.2 

0.6 

Lack  of  Motivation 

4 

1.7 

0.4 

Other  Personal 

1 

0.4 

0.1 

Unknown 

12 

5.2 

1.3 

*The  number  of  students  who  were  eliminated  prior  to  the 
start  of  lERW  flight  training. 

**The  total  number  of  students  eliminated  prior  to  or  during 
lERW  flight  training. 


Comments : 

Twenty-six  percent  of  all  students  eliminated  from  the  lERW 
program  were  eliminated  prior  to  the  start  of  flight  training. 

Resignation  was  given  as  the  cause  for  43.5%  of  the  pre-flight 
eliminations  and  medical/fitness  was  given  as  the  cause  of  18.5% 
of  the  pre-flight  eliminations.  No  data  were  recorded  on  the 
reasons  for  the  resignations  or  the  types  of  medical  problems  that 
led  to  the  eliminations  (only  4  students  were  eliminated  because 
they  failed  the  Army  Physical  Fitness  Test) .  Discussions  with 
SMEs  indicated  that  a  large  proportion  of  resignations  and  medical 
eliminations  are,  in  fact,  due  to  knowledge/skill  deficiencies  or 
behavioral  problems . 

Misconduct  and  failure  to  progress  were  given  as  the  causes  of 
elimination  in  10.3%  and  8.6%  of  the  cases,  respectively.  These 
numbers  probably  would  be  far  higher  if  the  true  causes  of 
resignations  and  medical  eliminations  were  known.  The  same  could 
be  said  for  the  causes  academic  (2.2%)  and  lack  of  motivation 
(1.7%) 

Together,  unit  recall  and  other  administrative  problems  were  given 
as  the  cause  of  9.5%  of  the  pre-flight  eliminations. 
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Table  F-2 


Causes  cited  for  setbacks  prior  to  the  start  of  lERW  flight 
training . 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

%  GROUP* 
(N=419) 

%  TOTAL** 
(N=4,672) 

Medical 

142 

33.9 

3.0 

Failure  to  Progress 

101 

24.1 

2.2 

Misconduct 

52 

12.4 

1.1 

Personal  Problems 

48 

11.5 

1.0 

Behind  Schedule 

22 

5.3 

0.5 

Physical  Fitness 

20 

4.8 

0.4 

Academic 

16 

3.8 

0.3 

At  t i tude /Mot i vat ion 

5 

1.2 

0.1 

Other 

13 

3.1 

0.3 

*The  number  of  students  who  were  setback  prior  to  the  start 
of  lERW  flight  training. 

**The  total  niomber  of  setbacks  prior  to  or  during  lERW  flight 
training. 


Comments : 

Setbacks  during  pre-flight  training  accounted  for  only  9%  of 
all  setbacks  that  occurred  during  lERW  training.  By 
comparison,  about  26%  of  all  lERW  eliminations  occurred 
during  pre- flight  training. 

The  first  eight  descriptors  listed  above  (excluding  "other") 
account  for  97%  of  all  the  causes  cited  for  pre-flight 
setbacks . 

Discussions  with  SMEs  indicated  that  most  of  the  medical- 
setbacks  during  pre- flight  training  are,  in  fact,  due  to 
knowledge /ski 11  deficiencies  or  behavioral  problems.  Hence, 
except  for  "personal  problems,"  "physical  fitness,"  and 
"other, "  the  descriptors  of  cause  listed  in  Table  F-2  imply  a 
knowledge /ski 11  deficiency  or  a  behavioral  problem. 
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Table  F-3 


Eliminations  during  lERW  flight  training  for  which  the  cited 
cause  was  a  ski 11 /knowledge  deficiency  or  behavioral  problem. 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

ss  GROUP* 
(N=341) 

%  TOTAL** 
{N=891) 

Flight  Deficiency 

274 

80.4 

30.8 

Misconduct 

58 

17.0 

6.5 

Academic 

7 

2.1 

0.8 

Accident 

2 

0.6 

0.2 

*The  number  of  students  who  were  eliminated  during  lERW 
flight  training  due  to  knowledge /ski 11  deficiencies  or 
behavioral  problems . 

**The  total  mamber  of  students  eliminated  prior  to  or  during 
lERW  flight  training. 


Comments : 

The  numbers  shown  in  Table  F-3  include  students  who  were 
clearly  eliminated  and  students  who  were  relieved  from  duty 
and  not  subsequently  reinstated. 


Clearly,  knowledge  or  skill  deficiency  (flight  deficiency, 
and  academic)  was  a  far  more  common  cause  of  eliminations 
during  lERW  flight  training  than  behavioral  problems 
(misconduct) .  Even  so,  some  form  of  misconduct  was  the  cause 
cited  for  17%  of  the  eliminations  in  this  group. 

There  is  no  reason  to  believe  that  test  instruments  could  not 
be  developed  to  detect  both  the  knowledge /ski 11  deficiencies 
and  the  behavioral  problems  that  lead  to  elimination  during 
the  flight  training  portion  of  lERW. 

It  is  noteworthy  that  academic  failure  accounts  for  only  2% 
of  the  eliminations  in  this  group. 
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Table  F-4 


Eliminations  during  AQC  flight  training  for  which  the  cited 
cause  was  a  ski 11 /knowledge  deficiency  or  behavioral  problem. 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

%  GROUP* 
(N=61) 

%  TOTAL** 
(N=210) 

Flight  Deficiency 

47 

77.0 

22.4 

Misconduct 

11 

18.0 

5.2 

Departed  Without  Clearing 

1 

1.6 

0.5 

Not  Selected  for  Promotion 

1 

1.6 

0.5 

Academic 

1 

1.6 

0.5 

*The  number  of  students  who  were  eliminated  during  AQC 
training  due  to  knowledge /ski 11  deficiencies  or  behavioral 
problems . 

**The  total  niomber  of  students  eliminated  during  AQC 
training . 


Comments : 

Flight  deficiency  was  cited  as  the  cause  of  77%  of  the  AQC 
eliminations  in  this  group  (knowledge,  skill,  or  behavior 
problems)  and  22.4%  of  all  AQC  eliminations.  Misconduct  was 
a  less  frequent  but  non-trivial  cause  of  AQC  eliminations. 

It  is  interesting  to  note  that  flight  deficiencies  accounted 
for  the  same  percentage  (77%)  of  the  lERW  eliminations  that 
were  due  to  knowledge,  skill,  or  behavior  problems. 

Although  academics  are  quite  difficult  for  most  of  the  AQC 
aircraft,  an  academic  failure  was  cited  as  the  cause  of  only 
1.6%  of  the  eliminations  in  this  group  and  only  0.5%  of  all 
AQC  eliminations . 
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Table  F-5 

Eliminations  during  lERW  flight  training  for  which  the  cited 
cause  was  a  factor  other  than  a  skill /knowledge  deficiency  or 
behavioral  problem. 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

%  GROUP* 
(N=326) 

%  TOTAL** 
(N=891) 

Resignation 

160 

49.1 

18.0 

Medical 

100 

30.7 

11.2 

Administrative  Problems 

30 

9.2 

3.4 

Body  Size  or  Fitness 

12 

3.7 

1.3 

Unit  Recall 

10 

3.1 

1.1 

Personal  Problems 

2 

0.6 

0.2 

Other 

12 

3,7 

1.3 

*The  number  of  students  who  were  eliminated  during  lERW 
training  due  to  causes  other  than  knowledge/skill 
deficiencies  or  behavioral  problems. 

**The  total  number  of  students  eliminated  during  lERW 
training . 


Comments : 

Resignation  was  cited  as  the  cause  of  18%  of  all  lERW 
eliminations  and  nearly  one-half  of  the  eliminations  in  this 
group.  The  reasons  for  the  resignations  were  not  recorded. 
Discussions  with  SMEs  knowledgeable  about  lERW  trainees 
indicate  that  resignations  are  the  result  of  poor  performance 
during  training,  a  fear  of  flying,  a  lack  of  motivation,  or  a 
combination  of  these  factors.  Hence,  the  factors  that 
contributed  to  resignations  appear  to  be  the  same  as  the 
factors  that  contributed  to  eliminations . 

Medical  was  cited  at  the  cause  of  30.7%  of  the  eliminations 
in  this  group.  The  SMEs  indicated  that  the  true  cause  of 
many  medical  eliminations  was  poor  performance  during 
training,  a  fear  of  flying,  or  a  lack  of  motivation. 
Apparently,  some  students  find  a  medical  elimination  to  be 
more  palatable  than  a  flight-deficiency  elimination  or  a 
resignation. 

Together,  unit  recall  and  other  administrative  problems 
accounted  for  12.3%  of  the  eliminations  in  this  group. 

It  is  interesting  to  note  that  7  students  were  eliminated 
because  they  were  too  tall  and  4  students  were  eliminated 
because  they  were  too  short . 
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Table  F-6 


Eliminations  during  AQC  flight  training  for  which  the  cited 
cause  was  a  factor  other  than  a  skill /knowledge  deficiency  or 
behavioral  problem. 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

%  GROUP* 
(N=148) 

%  TOTAL** 
(N=210) 

Unit  Recall 

39 

26.4 

18.6 

Resignation 

30 

20.3 

14.3 

Aircraft  Grounding 

20 

13.5 

9.5 

Lack  of  Prerequisites 

18 

12.2 

8.6 

Administrative  Problems 

17 

11.5 

8.1 

Medical 

15 

10.1 

7.1 

Death 

7 

4.7 

3.3 

Over  Weight 

2 

1.4 

1.0 

*The  number  of  students  who  were  eliminated  during  AQC 
training  due  to  causes  other  than  knowledge /ski 11 
deficiencies  or  behavioral  problems. 

**The  total  number  of  students  eliminated  during  AQC 
training. 


Comments : 


About  19%  of  all  AQC  eliminations  and  26.4%  of  the 
eliminations  in  this  group  are  due  to  unit  recall.  The 
reasons  for  the  unit  recall  are  not  known.  It  is  possible 
that  some  students  are  recalled  because  of  poor  performance 
in  training,  but  there  is  no  evidence  to  support  this  claim. 

About  14%  of  all  AQC  eliminations  and  20.3%  of  the 
eliminations  in  this  group  are  due  to  resignations. 
Discussions  with  SMEs  indicated  that  most  AQC  resignations 
are  due  to  poor  performance  in  training. 


It  seems  curious  that  8.6%  of  all  AQC  eliminations  were  due 
to  the  students'  lack  of  prerequisites.  It  also  is  of 
interest  that  aircraft  grounding  was  cited  as  the  cause  of 
9.5%  of  all  AQC  eliminations  and  13.5%  of  the  eliminations  in 
this  group. 

The  number  of  medical  eliminations  (7.1%  of  total)  that  were, 
in  fact,  due  to  poor  performance  in  training  is  not  known. 
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Table  F-7 


Setbacks  during  lERW  flight  training  for  which  the  cited 
cause  was  a  skill /knowledge  deficiency  or  behavioral  problem. 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

%  GROUP* 
{N=l,322) 

%  TOTAL** 
(N=4,672) 

Flight  Deficiency 

1211 

91.6 

25.9 

Misconduct 

89 

6.7 

1.9 

Academic 

13 

1.0 

0.3 

Company  Action 

5 

0.4 

0.1 

Motivation/Attitude 

2 

0.2 

0.0 

Other 

2 

0.2 

0.0 

*The  number  of  setbacks  during  lERW  training  due  to 
knowledge /ski 11  deficiencies  or  behavioral  problems. 
**The  total  number  of  setbacks  during  lERW  training. 


Comments : 

A  total  of  36  different  descriptors  were  cited  as  the  cause 
of  this  group  of  setbacks.  These  descriptors  were  classified 
into  the  6  categories  listed  above  in  Table  F-7 . 

Nearly  26%  of  all  lERW  setbacks  and  nearly  92%  of  the 
setbacks  in  this  group  were  due  to  a  flight  deficiency.  In 
contrast,  an  academic  deficiency  was  cited  as  the  cause  of 
very  few  lERW  setbacks  (0.3%  of  total). 

Misconduct  and  motivation/attitude  were  cited  as  the  causes 
of  a  small  but  non-trivial  parentage  (about  2%)  of  all  lERW 
setbacks.  It  is  probable  that  the  cause  descriptor  "company 
action"  indicates  some  type  of  misconduct. 
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Table  F-8 


Setbacks  during  AQC  flight  training  for  which  the  cited  cause 
was  a  skill /knowledge  deficiency  or  behavioral  problem. 


DESCRIPTOR 

OF  CAUSE 

NUMBER 

Ss  GROUP* 
(N=284) 

%  TOTAL** 
(N=581) 

Flight  Deficiency 

277 

97.5 

A1 .1 

Misconduct 

5 

00 

0.9 

Academic 

2 

0.7 

0.3 

*The  number  of  setbacks  during  AQC  training  due  to 
knowledge/skill  deficiencies  or  behavioral  problems. 
**The  total  number  of  setbacks  during  AQC  training. 


Comments : 

Flight  deficiency  was  the  cited  cause  of  47.7%  of  all  AQC 
setbacks  and  97.5%  of  the  setbacks  in  this  group.  All  other 
causes  of  setbacks  combined  account  for  only  1.2%  of  all  AQC 
setbacks  and  2.5%  of  the  setbacks  in  this  group. 
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Table  F-9 


Setbacks  during  lERW  flight  training  for  which  the  cited 
cause  was  a  factor  other  than  skill /knowledge  deficiency  or 
behavioral  problem. 


DESCRIPTOR 

NUMBER 

%  GROUP* 

%  TOTAL** 

OF  CAUSE 

{N=2.925) 

(N=4,672) 

IP  Shortage 

1463 

50.0 

31.3 

Medical 

829 

28.3 

17.7 

Personal  Problems 

309 

10.6 

6.6 

Behind  Schedule 

151 

5.2 

3.2 

Weather /Maintenance 

27 

0.9 

0.6 

Other  Administrative 

26 

0.9 

0.6 

Physical  Fitness 

20 

0.7 

0.4 

Aircraft  Shortage 

18 

0.6 

0.4 

Other 

82 

2.8 

1.8 

*The  number  of  setbacks  during  lERW  training  due  to  a  factor 
other  than  knowledge/ skill  deficiencies  or  behavioral 
problems . 

**The  total  number  of  setbacks  during  lERW  training. 

Comments : 

A  total  of  77  different  descriptors  were  cited  as  the  cause 
of  this  group  of  setbacks.  These  descriptors  were  classified 
into  the  9  categories  listed  above  in  Table  F-9. 

The  two  most  frequently  cited  causes  of  lERW  setbacks  in  this 
group  are  IP  Shortage  (50.0%)  and  medical  (28.3%).  SMEs 
indicated  that  many  of  the  medical  setbacks  were,  in  fact, 
due  to  knowledge /ski 11  deficiencies  or  behavioral  problems. 

A  variety  of  personal  problems  were  cited  as  the  cause  of  a 
substantial  percentage  of  setbacks  (6.6%  of  total  and  10.6% 
of  group) .  Examples  of  the  descriptors  include  emergency 
leave,  leave,  family  emergency,  and  personal  (unspecified)  . 

A  descriptor  indicating  that  the  student  was  behind  the 
training  schedule  was  cited  as  the  cause  of  3.2%  of  all 
setbacks  and  5.2%  of  the  setbacks  in  this  group.  The  reasons 
why  the  student  was  behind  schedule  were  not  reported. 

Physical  fitness  (lack  of) ,  other  administrative  problems, 
weather  and  maintenance  delays,  and  aircraft  shortages 
accounted  for  a  small  but  non-trivial  percentage  of  the 
setbacks . 
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Table  F-10 


Setbacks  during  AQC  flight  training  for  which  the  cited  cause 
was  a  factor  other  than  skill /knowledge  deficiency  or 
behavioral  problem. 


DESCRIPTOR 

NUMBER 

%  GROUP* 

%  TOTAL** 

OF  CAUSE 

{N=297) 

(N=581) 

Medical 

118 

39.7 

20.3 

Personal  Problems 

61 

20.5 

10.5 

Behind  Schedule 

47 

15.8 

8.1 

IP  Shortage 

33 

11.1 

5.7 

Weather  maintenance 

30 

10.1 

5.2 

Other  Administrative 

5 

1.7 

0.9 

Other 

3 

1.0 

0.5 

*The  number  of  setbacks  during  AQC  training  due  to  a  factor 
other  than  knowledge/ skill  deficiencies  or  behavioral 
problems . 

**The  total  number  of  setbacks  during  AQC  training. 


Comments : 

A  total  of  21  different  descriptors  were  cited  as  the  cause 
of  this  group  of  setbacks.  These  descriptors  were  classified 
into  the  7  categories  listed  above  in  Table  F-10. 

Medical  was  cited  as  the  cause  of  20.3%  of  all  AQC  setbacks 
and  39.7%  of  the  setbacks  in  this  group.  SMEs  stated  that 
most  of  the  medical  setbacks  during  AQC  are,  in  fact,  due  to 
a  temporary  medical  problem  (e.g.,  colds,  ear  infections, 
clogged  sinus  tracts) .  Even  so,  it  is  probable  that  some 
students  who  are  not  progressing  as  rapidly  as  their  peers 
may  claim  a  medical  problem  in  order  to  gain  additional  time 
for  study  or  practice. 

Leave  for  emergencies  and  other  personal  problems  were  cited 
as  the  cause  of  10.5%  of  all  AQC  setbacks  and  20.5%  of  the 
setbacks  in  this  group. 

Although  "behind  Syllabus"  was  cited  as  the  cause  of  8.1%  of 
all  AQC  setbacks,  the  reason  the  student  was  behind  syllabus 
was  not  reported. 

IP  Shortage  was  cited  as  the  cause  of  only  5.7%  of  all  AQC 
setbacks  (vs.  31.3%  of  all  lERW  setbacks). 
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Figure  F-1.  Frequency  of  four  causes  cited  for 
eliminations  prior  to  the  start  of  lERW  flight  training,  by 
year. 


1989  1990  1991  1992 

Year 


1993 


1994 


Comments : 

Only  one  pre-flight  elimination  was  reported  for  1995. 
This  is  surprising  in  light  of  the  large  number  of 
pre-flight  eliminations  reported  for  1994.  This  finding 
may  be  an  artifact  of  the  data  reporting  or  data  coding 
system. 
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Year 

Figure  F-2 .  Frequency  of  four  causes  cited  for  setbacks 
prior  to  the  start  of  lERW  flight  training. 


Comments : 

The  four  causes  of  setbacks  shown  in  Figure  F-2  account 
for  91%  of  all  the  setbacks  that  occurred  prior  to  the 
start  of  lERW  flight  training. 

The  small  number  of  setbacks  shown  for  1995  may. be  an 
artifact  stemming  from  a  change  in  the  reporting  or 
coding  system. 
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Figure  F-3 .  Frequency  of  four  causes  cited  for 
eliminations  that  occurred  during  lERW  flight  training. 


Comments : 

The  four  causes  of  eliminations  shown  in  Figure  F-3 
account  for  89%  of  all  the  eliminations  that  occurred 
during  lERW  flight  training. 
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Figure  F-4 .  Frequency  of  four  causes  cited  for  setbacks 
that  occurred  during  lERW  flight  training. 
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Comments : 

The  four  causes  of  setbacks  shown  in  Figure  F-4  account 
for  90%  of  all  the  setbacks  that  occurred  during  lERW 
flight  training. 
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Figure  F-5.  Frequency  of  four  causes  cited  for 
eliminations  that  occurred  during  AQC  training. 


Comments : 

The  four  causes  of  eliminations  shown  in  Figure  F-5 
account  for  only  46%  of  all  the  eliminations  that 
occurred  during  AQC  training.  In  contrast,  the  same 
four  causes  account  for  90%  of  all  the  eliminations  that 
occurred  during  lERW  flight  training.  A  large  portion 
of  AQC  eliminations  stem  from  administrative  problems 
such  as  unit  recall,  administrative  scheduling  problems, 
administrative  errors,  and  aircraft  groundings. 
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Figure  F-6.  Frequency  of  four  causes  cited  for  setbacks 
that  occurred  during  AQC  training. 


Comments : 

The  four  causes  of  setbacks  shown  in  Figure  F-6  account 
for  84%  of  the  setbacks  that  occurred  during  AQC 
training.  It  is  noteworthy  that  IP  shortage  is  cited  as 
the  cause  of  relatively  fewer  AQC  setbacks  than  lERW 
setbacks . 
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Appendix  G 

Results  of  Cut-Score  Analysis 


Comments : 

The  figures  in  Appendix  G  show  cumulative  distributions  of 
FAST  scores  for  (a)  lERW  eliminees,  (b)  AQC  eliminees,  (c) 
lERW  non-eliminees  who  experienced  one  or  more  setbacks 
during  training,  and  (d)  AQC  non-eliminees  who  experienced 
one  or  more  setbacks  during  training.  Separate  distributions 
are  shown  for  white  males  and  non-white  males.  The  sample 
size  for  individual  racial  groups  (e.g..  Black,  Hispanic, 
Asian,  American  Indian)  was  too  small  to  support  reliable 
cumulative  distributions  by  racial  group. 

A  cumulative  distribution  is  shown  for  female  lERW  students 
that  were  either  eliminated  or  experienced  one  or  more 
setbacks.  The  female  sample  size  was  too  small  to  show 
cumulative  distributions  as  a  function  of  type  of  training 
event  (elimination  vs.  setback)  or  race  (white  vs.  non¬ 
white)  . 

In  every  figure,  a  cumulative  distribution  is  shown  for  a 
comparison  group.  The  comparison  groups  consists  of 
individuals  who  (a)  completed  training  (lERW  or  AQC)  with  no 
setbacks  and  (b)  have  the  same  gender  and  race  (white  or  non¬ 
white)  as  the  eliminees  or  setbacks  whose  cumulative 
distribution  is  shown  in  the  figure. 

The  FAST  score  was  not  available  for  every  individual  in  the 
database  who  was  eliminated  or  who  experienced  one  or  more 
setbacks.  Even  so,  the  sample  size  for  all  the  cumulative 
distributions  shown  in  the  following  figures  is  large  enough 
to  be  reliable. 

The  group  of  eliminees  included  individuals  who  were 
eliminated  and  individuals  who  were  relieved  and  not 
reinstated.  The  group  of  setbacks  include  individuals  w;ho 
were  setback,  turned  back,  or  relieved  and  reinstated. 
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Figure  G-1.  Cumulative  distribution  of  FAST  scores  for 
white  male  lERW-eliminees  and  white  male  comparison  group 
meinbers.  Includes  individuals  who  were  enrolled  in  the  lERW 
training  program  during  the  period  1988-1995. 


Comments : 

The  purpose  of  the  cumulative  distributions  in  Figure  G-1 
and  all  that  follow  is  to  show  the  result  of  increasing  the 
FAST  cut  score.  For  example,  consider  the  effect  of 
increasing  the  cut  score  to  100.  It  can  be  seen  that  a  cut 
score  of  100  would  have  excluded  (a)  about  11%  of  the 
individuals  who  completed  lERW  successfully,  with  no 
setbacks  (comparison  curve) ,  and  (b)  about  23%  of  the 
individuals  who  were  eliminated  (eliminations  curve) . 
Similarly,  increasing  the  cut  score  to  110  would  have 
excluded  about  29%  of  the  comparison  group  members  and  47% 
of  the  eliminees . 

The  two  curves  diverge  as  FAST  score  increases;  this 
indicates  that  the  relative  benefits  of  increasing  cut 
score  tends  to  become  greater  cut  score  increases . 


G-2 


Figure  G-2 .  Cumulative  distribution  of  FAST  scores  for 
non-white  male  lERW-eliminees  and  white  male  comparison 
group  members.  Includes  individuals  who  were  enrolled  in  the 
lERW  training  program  during  the  period  1988-1995. 


Comments : 

Increasing  cut  score  to  100  would  have  excluded  about  45%  of 
the  non-white  eliminees,  but  at  a  cost  of  about  23%  of  the 
non-white  males  who  completed  lERW  with  no  setbacks. 

The  two  curves  tend  to  converge  as  FAST  score  increases . 

This  indicates  that  the  relative  benefits  of  increasing  cut 
score  becomes  smaller  as  cut  score  increases . 
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Figure  G-3 .  Cumulative  distributions  of  FAST  scores  for 
white  male  lERW-eliminees  as  a  function  of  the  cause 
cited  for  elimination.  Includes  individuals  who  were 
enrolled  in  the  lERW  training  program  during  the  period 
1988-1995. 


Comments : 

The  cumulative  distribution  for  the  individuals 
eliminated  because  of  a  skill  deficiency  is  quite 
similar  to  the  cumulative  distribution  for  the 
individuals  who  were  eliminated  for  other  reasons . 
However,  the  distributions  of  both  groups  of  eliminees 
are  quite  different  from  the  comparison  group 
distribution.  This  finding  supports  the  claims  of  SMEs 
that  many  of  the  individuals  eliminated  for  other 
reasons  (e.g.,  medical,  resignation,  personal)  were,  in 
fact,  eliminated  because  of  skill  deficiencies  or 
behavioral  problems . 
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Figure  G-4.  Cumulative  distributions  of  FAST  scores  for 
non-white  male  lERW-eliminees  with  individuals  grouped  in 
terms  of  the  cited  cause  of  the  elimination.  Includes 
individuals  who  were  enrolled  in  the  lERW  training 
program  during  the  period  1988-1995. 


Comments : 

The  cumulative  distribution  for  the  individuals 
eliminated  because  of  a  skill  deficiency  is  quite  ' 
different  from  the  cumulative  distribution  for  the 
individuals  who  were  eliminated  for  other  reasons .  This 
finding  suggests  that  many  of  the  non -white  males 
eliminated  for  other  reasons  are,  in  fact,  eliminated 
for  reasons  other  than  a  skill  deficiency  or  a 
behavioral  problem.  In  this  respect,  non-white  male 
eliminees  differ  from  white  male  eliminees. 
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Figure  G-5.  Cumulative  distributions  of  FAST  scores  for 
white  males  with  lERW  setbacks  and  white  males  with  no 
lERW  setbacks.  Includes  individuals  who  were  enrolled 
in  the  lERW  training  program  during  the  period 
1988-1995. 


Comments : 

The  c\amulative  distribution  for  individuals  with  one  or 
more  lERW  setbacks  is  only  slightly  different  from  the 
cumulative  distribution  of  individuals  with  no  lERW 
setbacks.  Hence,  increasing  cut  score  would  exclude 
only  5-8%  more  individuals  with  setbacks  than 
individuals  who  completed  lERW  training  with  no 
setbacks . 
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Figure  G-6.  Cumulative  distributions  of  FAST  scores  for 
non-white  males  with  lERW  setbacks  and  non-white  males 
with  no  lERW  setbacks  Includes  individuals  who  were 
enrolled  in  the  lERW  training  program  during  the  period 
1988-1995. 


Comments : 

As  was  true  for  white  males, the  cumulative  distribution 
for  non-white  males  with  one  or  more  lERW  setbacks  is 
only  slightly  different  from  the  cumulative  distribution 
for  non-white  males  with  no  lERW  setbacks .  Increasing 
cut  score  would  exclude  only  5-8%  more  individuals  with 
setbacks  than  individuals  who  completed  lERW  training 
with  no  setbacks . 


Figure  G-7 .  Cumulative  distributions  of  FAST  scores  for 
white  males  with  one  or  more  lERW  setbacks,  with 
individuals  grouped  in  terms  of  the  cited  cause  of  the 
setback.  Includes  individuals  who  were  enrolled  in  the 
lERW  training  program  during  the  period  1988-1995. 


Comments : 

The  cumulative  distribution  of  individuals  whose  setback 
was  due  to  a  skill  deficiency  or  behavioral  problem 
differs  substantially  from  the  distributions  of  the 
other  two  groups.  This  means  that  increasing  cut  score 
will  exclude  (a)  10-15%  more  students  with 
skill /behavior  setbacks  than  students  with  no  setbacks 
and  (b)  7-15%  more  students  with  skill /behavior  setbacks 
than  students  with  setbacks  caused  by  other  factors . 

The  curves  tend  to  diverge  as  FAST  score  increases . 

This  means  that  the  benefits  tend  to  become  greater  as 
the  cut  score  is  increased. 
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Figure  G-8.  Cumulative  distributions  of  FAST  scores  for 
AQC  eliminees  who  have  been  grouped  in  terms  of  the 
cited  cause  of  the  elimination.  Includes  individuals 
who  were  enrolled  in  an  AQC  course  during  the  period 
1988-1995. 


Comments ; 

The  above  distributions  show  little  or  no  benefit  of 
increasing  cut  score  to  any  value  below  about  108.  For 
cut  scores  higher  than  108,  between  10%  and  18%  more 
eliminees  (skill /behavior)  would  be  excluded  than 
members  of  the  comparison  group.  It  is  noteworthy  that 
increasing  cut  score  to  any  value  between  100  and  120 
would  exclude  about  the  same  percentage  of  the 
comparison  group  as  individuals  eliminated  for  reasons 
other  than  a  skill  or  behavior  problem.  Because  of  the 
small  number  of  ACQ  eliminees,  however,  the  eliminee 
distributions  shown  in  Figure  G-8  cannot  be  considered 
to  be  highly  reliable. 
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Figure  G-9 .  Cumulative  distributions  of  FAST  scores  for 
AQC  students  with  one  or  more  setbacks  who  have  been 
grouped  in  terms  of  the  cited  cause  of  the  elimination. 
Includes  individuals  who  were  enrolled  in  an  AQC  course 
during  the  period  1988-1995. 


Comments : 

The  distributions  for  AQC  students  with  setbacks  show 
that  cut  score  would  have  to  be  increased  to  about  108 
to  exclude  a  larger  percentage  of  AQC  students  with 
setbacks  than  studencts  in  the  ACQ  comparison  group. 
Even  for  values  above  108,  the  difference  is  10%  or 
less . 
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Figure  G-10.  Cxomulative  distributions  of  FAST  scores 
for  females  with  lERW  setbacks  or  eliminations  and 
females  who  completed  lERW  training  with  no  setbacks. 
Includes  individuals  who  were  enrolled  in  the  lERW 
training  program  during  the  period  1988-1995. 


Comments : 

Because  of  the  small  sample  size,  the  cumulative 
distributions  shown  above  are  far  more  irregular  than 
the  cumulative  distributions  depicted  in  Figures  G1-G9 . 
Even  so,  the  distributions  shown  in  Figure  G-10  provide 
no  indication  that  the  FAST's  predictive  validity  is.no 
better  or  worse  for  female  lERW  students  than  for  male 
lERW  students . 
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